Skip to content

fix(ltx-2.5): the audio VAE's norm_eps was read by NO arm, and the keyframe refusal named a reason L11 removed (#560) - #563

Merged
localai-bot merged 7 commits into
row/MODEL-DIFFUSION-LTX25from
row/LTX25-EPS-PIN
Aug 13, 2026
Merged

fix(ltx-2.5): the audio VAE's norm_eps was read by NO arm, and the keyframe refusal named a reason L11 removed (#560)#563
localai-bot merged 7 commits into
row/MODEL-DIFFUSION-LTX25from
row/LTX25-EPS-PIN

Conversation

@localai-bot

@localai-bot localai-bot commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Closes the two LOW findings from the phase-L11 review, the pre-existing half of
the first that L4 left behind, and two further instances the required sweep
turned up.

Issue: #560. Row: MODEL-DIFFUSION-LTX25 (#435).
Base: row/MODEL-DIFFUSION-LTX25 @ 127d29156.

F1 — norm_eps was a constant NO ARM READ, on both audio VAE halves

Ltx2AudioEncoderConfig::norm_eps was missing from the pin list whose own
comment says "Adding a new constant without adding it to this list reopens the
hole"
, and Ltx2AudioDecoderConfig::norm_eps had the identical hole from L4.

RED before, both mutated 1e-6 -> 1e-4 (100x), on the pre-change tree:

[doctest] test cases:   33 |   33 passed | 0 failed | 0 skipped
[doctest] assertions: 2602 | 2602 passed | 0 failed |
[doctest] Status: SUCCESS!

The cause is worse than inertness: every audio arm ran norm_type = kPixel, so
ApplyNorm never entered the GroupNorm branch and the constant was never READ.
A pin alone would have recorded that rather than removed it, so this takes the
option §7.0(a) asks for and adds two GOLDEN ARMS that execute the branch,
generated from the pinned oracle.

norm_type = group is the DEFAULT of both AudioEncoder.__init__ and
AudioDecoder.__init__ and
CORRECTED — see "Five inaccurate upstream claims"
below.
norm_type = group is indeed the declared default
(audio_vae/audio_vae.py:82), but its PAIRED default is
causality_axis = WIDTH (:83), and ResnetBlock.__init__ raises
ValueError: Causal ResnetBlock with GroupNorm is not supported for any
causality_axis != NONE (audio_vae/resnet.py:130-131) — so constructing either
class on pure defaults RAISES, and "the default" does not establish what this
sentence used it to establish. What survives, and is all the arms need, is that
the combination is legal at causality_axis: none, so this is a configuration a
checkpoint may declare, not a hypothetical.

The new arms are numerical, not assertional. The same 100x mutation now moves
the decoder golden by 1.13e-3 and the encoder golden by 5.18e-3 against a
5e-6 band, i.e. 225x and 1036x over:

TEST CASE:  ltx2 vae: the GROUP-NORM audio decoder matches upstream ltx_core
  values: CHECK( 0.00112667 <= 5e-06 )
TEST CASE:  ltx2 vae: the GROUP-NORM audio encoder matches upstream ltx_core
  values: CHECK( 0.00518311 <= 5e-06 )

Two reduced dimensions move on those arms and both are FORCED: ch becomes 32
because build_normalization_layer hardcodes num_groups=32 CORRECTED —
see "Five inaccurate upstream claims" below
num_groups: int = 32 is
build_normalization_layer's own keyword DEFAULT (common/normalization.py:44),
which every audio VAE call site takes rather than a hardcoded literal — and
torch's GroupNorm refuses a channel count 32 does not divide, and z_channels
becomes 16 because PerChannelStatistics indexes the patchified (c, f) axis.
Nothing else changes.

The goldens regenerate byte-identically outside the two new sections: the
generator was run against the pinned oracle before and after, and the diff is
276 added lines, zero removed.

F2 — a refusal whose stated reason was false

The keyframe / reference refusal still read "phase L4 ported the DECODER only".
L11 ported the encoder. The effect was unchanged and correct, but a stale reason
sends the next reader to build something that already exists. It now names what
is actually missing: no VAE_ENCODER_COMFY_KEYS_FILTER / VideoEncoderConfigurator
loader path puts encoder weights in memory, and ImageConditioner.resolve_crf's
H.264 default_image_crf round trip is not ported.

The test guarding it accepted either message, so it now asserts both facts and
goes RED on the old text:

CHECK( msg.find("VAE_ENCODER_COMFY_KEYS_FILTER") != std::string::npos ) is NOT correct!
CHECK( msg.find("default_image_crf") != std::string::npos ) is NOT correct!
[doctest] test cases:  17 |  16 passed | 1 failed | 0 skipped

The sweep, which is the point

A 20-constant mutation sweep over every stabilizing epsilon, clamp and floor in
ltx2*.h / ltx2*.cpp, each mutated alone and rebuilt, found two more:

  • Ltx2DitParams::norm_eps is INVISIBLE. 1e-6 -> 1e-4 left ALL SIX LTX
    suites green. Not because it is unused, but because every arm passes it
    explicitly through ReducedParams, so nothing reads the FIELD DEFAULT — which
    is live code, since ParseLtx2DitParams falls back to it exactly as upstream's
    config.get("norm_eps", 1e-06) does. Pinned where it binds, in the parse case.
  • Ltx2AudioEncoderConfig::pixel_norm_eps was reachable but outside the
    "two PixelNorm epsilons stay different" case, which only ever held the DECODER
    pair. Extended to the encoder pair L11 added.

kLtx2RmsNorm2dEps is caught by its PIN ALONE, and kLtx2EncoderApproxLnZero has
no consumer at all because upstream concatenates it and immediately discards it
with torch.chunk(...)[0] (video_vae.py:326-336).

Review repairs (757a7bb24)

A constant this PR called "correctly unreachable" is read on every arm.
Ltx2ConvVideoDecoderConfig::norm_eps was classified "pinned, correctly
unreachable — upstream discards the value"
on the reasoning that the decoder runs
norm_layer = PIXEL_NORM. That reasoning is wrong. ResnetBlock3D.__init__
builds norm3 = nn.GroupNorm(num_groups=1, num_channels=in_channels, eps=eps)
whenever in_channels != out_channels (video_vae/resnet.py:93-97) regardless
of norm_layer
, and applies it at resnet.py:178. Every res_x_y block reads
it, and the shipped section-5 fixture has one.

What was true was a sensitivity property of ONE FIXTURE, not invisibility: five
blocks deep the norm3 variance is ~0.2, so 1e-6 -> 1e-4 moves 1.8e-6 (under the
5e-6 band) while 1e-6 -> 1.0 moves 1.6e-2. A pin records that hole rather than
removing it, so section 5d is a new golden arm that removes it — one res_x_y
block behind conv_in, latent at a tenth scale, leaving norm3 a variance of
~5e-3. RED, mutating the epsilon at its POINT OF USE so only the numerics speak:

1e-6 -> 1e-5 (smallest realistic typo):
  values: CHECK( 0.000919342 <= 5e-06 )
1e-6 -> 0 (removing it; no other arm can see this):
  values: CHECK( 0.000102103 <= 5e-06 )
both: [doctest] test cases: 36 | 35 passed | 1 failed  Status: FAILURE!

One case fires out of 36 in both — the five arms that already execute norm3 stay
green through a 10x error and through deleting the constant. GREEN restored at
36/3039. The port's 1.021e-4 under eps=0 matches the oracle's own 1.024e-4.

A sixth constant, honestly labelled. Ltx2AttentionArgs::norm_eps
(attention.py:485) is assigned by every call site, so a 10^6 mutation leaves
every LTX suite green except the new pin. Never READ rather than never binding —
a latent trap, not coverage, and the comment says so.

Five inaccurate upstream claims. norm_type = group "is the DEFAULT of both
AudioEncoder.__init__ and AudioDecoder.__init__" is misleading: the paired
default is causality_axis = WIDTH, and constructing either on pure defaults
RAISES ValueError: Causal ResnetBlock with GroupNorm is not supported
(resnet.py:130-131) — verified by construction. Anchors audio_vae.py:81/:295
corrected to :82/:294. Also: num_groups=32 is build_normalization_layer's
own keyword default (normalization.py:44), not a hardcoded literal; and
resolve_crf only FILLS IN default_image_crf (blocks.py:977-983) — the H.264
round trip is preprocess (media_io/decode.py:413-435) and is skipped at
crf == 0.

Gates

Clean Release CPU build, -DVLLM_CPP_CUDA=OFF:

suite before after Status
test_ltx2 29/1615 30/1627 SUCCESS
test_ltx2_vae 33/2602 36/3039 SUCCESS
test_ltx2_text_encoder 17/3350 17/3350 SUCCESS
test_ltx2_pipeline 35/2358 35/2358 SUCCESS
test_ltx2_loader 20/2363 20/2363 SUCCESS
test_ltx2_video 17/170 17/172 SUCCESS
test_ops_attention_cross 9/32 9/32 SUCCESS
test_video_engine 11/254 11/254 SUCCESS
test_capi 55/505 55/505 SUCCESS
test_minimax_h3 79/57395 79/57395 SUCCESS
test_minimax_h3_video_fold 6/137 6/137 SUCCESS
test_ltx2_device - 13/498 SUCCESS

CORRECTED. An earlier revision of this section claimed "ctest -j4: 399/399
passed, 0 failed (399 rather than 401 is this CPU-only configuration's registered
count, not a dropped test)"
. That reconciliation was false and was never checked:
ctest -N reports Total Tests: 401. Nothing was dropped, but the explanation
was manufactured, which is exactly the shape of reasoning that would have hidden a
real drop. The honest gate, on a clean from-scratch Release CPU build of
757a7bb24:

$ ctest --test-dir build -N
Total Tests: 401

$ ctest --test-dir build -j4
100% tests passed, 0 tests failed out of 401

One earlier -j4 run showed test_engine_core_proc failing; it passes serially
and passed on the repeat -j4 run, i.e. the known -j starvation, not a
regression. This branch adds no CMake test target, so 401 is unchanged from base.

Every count that moved is an addition: +10 assertions in test_ltx2 is the DiT
pin and +1 case / +2 the Ltx2AttentionArgs pin, +396 in test_ltx2_vae is the
two group-norm arms and their manifests and +1 case / +41 the norm_eps-binding
arm, +2 in test_ltx2_video is the refusal's two new assertions.

Known, not from this change: doc-checkpoint range fails on b0aa475a3 and
d67f8125e, which predate this branch. device-leakage CI (#553) and
test_ltx2_device ordering (#516) are likewise pre-existing and untouched.

Owed to the operator: #560 needs its row in the .agents/roadmap_v1.md issue
table and a line in the row spec; those surfaces are the operator's and were not
edited here.

Re-review repair (d45bcb5fb) — the pin list shipped three more wrong verdicts

The re-review returned FAIL narrowly, on record accuracy rather than on
behaviour. All eight prior findings stayed CLOSED and were independently
reproduced; nothing about the norm_eps arm or the upstream revision pin
changed. But this PR exists to stop constants carrying a wrong reachability
verdict, and it shipped three of its own — which, given the deliverable is an
accurate record, IS the defect.

N-1 (blocking) — two constants listed as invisible are numerically gated. The
pin case opened with "Each of these was mutated with every golden staying
green"
, false for both encoder entries. Re-measured here, mutating the FIELD
DEFAULTS (no arm overrides them):

Ltx2ConvVideoEncoderConfig::norm_eps       1e-6 -> 1e-4   RED, 2 goldens
  CHECK( 4.38839e-05 <= 5e-06 )   the video ENCODER (*_res family)
  CHECK( 4.38839e-05 <= 5e-06 )   the video encoder CROPS a frame count ...

Ltx2ConvVideoEncoderConfig::pixel_norm_eps 1e-8 -> 1e-6   RED, 4 goldens
  CHECK( 1.02744e-05  <= 5e-06 )  the video ENCODER (*_res family)
  CHECK( 0.000175595  <= 5e-06 )  ... (strided convs, per_channel, reflect)
  CHECK( 1.02744e-05  <= 5e-06 )  the video encoder CROPS a frame count ...
  CHECK( 8.10623e-06  <= 5e-06 )  video ENCODER temporal causality ...

The cause is F-1's, reaching the other half through the SAME LINE: the encoder
shares ResnetBlock3d, so ltx2_video_vae.cpp:1051,1056 call what the decoder
calls at :693,700 and read norm3 at :405. Forcing :405 to 1.0 reds those
two encoder goldens at 0.150858, which is what IDENTIFIES norm3 as the reader
rather than inferring it — norm_layer is kPixelNorm on both encoder arms, so
neither ApplyNorm nor conv_norm_out (:1081-1087) enters a GroupNorm branch.
Arm B has no res_x_y, so no norm3, and stays green: the coverage is real but
PARTIAL, which is why both pins stay. Coverage was better than recorded, not
worse — and a list whose membership claim is wrong in the safe direction is one
nobody can trust in the unsafe one. The class statement is now per-entry,
quantified, and states that the mutation was run.

N-2 — the repair's own new arm falsified a line it left standing.
ltx2_video_vae.h still recorded Ltx2ConvVideoDecoderConfig::pixel_norm_eps 1e-8 -> 1e-6 green under "EVERY golden staying green". With section 5d present
that mutation REDS: CHECK( 0.000169305 <= 5e-06 ). The low-scale latent built to
make norm_eps a first-order term made its neighbour one too.

N-3 — the encoder half gets the decoder's note.
ltx2_video_vae_encoder.h:121-124 said only "a field here only so the gate can
pin it"
; true, and incomplete for the identical code path. One line cannot be
live for one caller and dead for the other.

N-4 — the qualifier F-8 exists to add is now gated. The refusal test checked
only VAE_ENCODER_COMFY_KEYS_FILTER and default_image_crf, so deleting the
crf == 0 qualifier again would not have gone red. Upstream preprocess returns
the image untouched at crf == 0 (media_io/decode.py:425-426, within
:413-435; :427 is a different early return, the degenerate-size guard —
corrected in the round-3 section below).
RED proof, with the qualifier removed from the message:

TEST CASE:  ltx2 video: keyframe and reference conditioning is refused by name
tests/vllm/multimodal/test_ltx2_video.cpp:674: ERROR:
  CHECK( msg.find("unless that CRF is 0") != std::string::npos ) is NOT correct!
[doctest] test cases:  17 |  16 passed | 1 failed | 0 skipped
[doctest] assertions: 173 | 172 passed | 1 failed |

N-5 / N-6 / N-7. gen-ltx2-vae-goldens.py hardcoded 10 * 5e-6 while the
suite applies kLtx2GoldenTol — two definitions of one number in two languages,
so a widened C++ band would leave the generator certifying arms against a band
nobody uses. It now PARSES kLtx2GoldenTol out of the suite and dies unless it
finds exactly one definition. docs/USAGE.md:335 had grown to 119 chars inside a
79-column paragraph; re-wrapped, no wording changed. And the F1 section's two
retracted upstream claims now carry in-place CORRECTED markers, so a reader of
that section alone no longer gets a false statement.

Gate for d45bcb5fb

Clean from-scratch Release CPU build, -DVLLM_CPP_CUDA=OFF:

BUILD_EXIT=0                      (no "No space left" / "BFD assertion" line)
$ ctest --test-dir build-eps -N
Total Tests: 401
$ ctest --test-dir build-eps -j4
99% tests passed, 1 tests failed out of 401 -- test_engine_core_proc
$ ctest --test-dir build-eps -R '^test_engine_core_proc$'
100% tests passed, 0 tests failed out of 1     (the known -j starvation)
suite before after Status
test_ltx2_vae 36/3039 36/3039 SUCCESS (comment-only)
test_ltx2_video 17/172 17/173 SUCCESS (+1, the N-4 CHECK)

Every mutation above was applied to the working tree, built, run, and REVERTED,
with the file's md5 checked back to its pre-mutation value each time.

Round-3 repair (ef83d9400) — one entry over, the list still said green

Round 3 confirmed all seven of round 2's repairs reproduce to the digit, including
the partial-coverage claim, and failed on one new instance of the defect this PR
exists to remove: a record calling a constant unreachable when a golden reds on it.

R-1 (blocking) — kLtx2BweMelLogClamp is not invisible. ltx2_video_vae.h:108-110
listed it under "these three left EVERY golden green". Reproduced here before the
text was touched, by mutating ltx2_audio_vae.h:212 1e-5 -> 1e-8:

tests/vllm/models/test_ltx2_vae.cpp:1400: ERROR:
  CHECK( err <= kLtx2GoldenTol ) is NOT correct!
  values: CHECK( 0.144965 <= 5e-06 )
  logged: saturated-clamp BWE max|diff| = 0.144965
tests/vllm/models/test_ltx2_vae.cpp:1279: ERROR:
  CHECK( vllm::kLtx2BweMelLogClamp == doctest::Approx(1e-5)... ) is NOT correct!
  values: CHECK( 1e-08 == Approx( 1e-05 ) )

[doctest] test cases:   36 |   34 passed | 2 failed | 0 skipped
[doctest] assertions: 3039 | 3037 passed | 2 failed |
[doctest] Status: FAILURE!            exit code 1

Restored byte-for-byte, md5 fe861901ef8f18fbbc9e6caedf513d1b before and after.

The saturating arm that reds it landed at 93329b119, and git log -S "left EVERY golden green" returns only d45bcb5fb — the phrase was written over an arm that
already existed. The tree already contradicted itself twice, at ltx2_video_vae.h:136-139
and test_ltx2_vae.cpp:1274-1278. The entry is moved out with the number that proves
it, as round 2 moved pixel_norm_eps. What made it look invisible is SCALE, not the
constant's nature: the ordinary arm's raw mel minimum is ~4.4e-3, so the reachable arm
attenuates mel_basis by 1e-4 until every bin saturates.

R-1b — the same false claim sat at the constant's own definition. ltx2_audio_vae.h:206
called it "the member of the invisible-constant class" and said the mutation "leaves
every tensor golden green"
. Both sentences shipped in 93329b119, the commit that added
the golden refuting them, so fixing only the list would have handed round 4 the identical
finding one file over. Its citation was also off by one: the clamp is vocoder.py:515,
not :516 (:516 is the return) — verified at the pinned checkout, and already cited
correctly by test_ltx2_vae.cpp:1273.

R-2 — the CRF citation pointed at a neighbouring early return. At LTX-2 fd4ded7f:

425:     if crf == 0:
426:         return image
427:     if min(image.shape[0], image.shape[1]) < 2:
428:         return image

:427 is the degenerate-size guard, a different early return. The enclosing range and
the behavioural claim were both correct, so only the pinpoint moves — but this is the
worst place to be off by two, because a reader following it lands on the size guard and
concludes the crf == 0 claim is unsupported. Now cited as :425-426, naming :427-428
as the guard it is not, here and in N-4 above. That guard is also a second case where
upstream does not re-compress, which docs/USAGE.md claimed happened whenever the CRF is
nonzero, with no other exception; qualified there too.

Optional item — a shared line is not an argument for liveness. ltx2_video_vae_encoder.h:130-132
argued "It is ONE line in the port for both halves ... so it cannot be live for one and
dead for the other"
. The conclusion holds; the inference does not, and round 3's
:405 -> 1.0 probe is the counterexample. :405 sits behind the input.channels != out_channels guard at :400, so entering ResnetBlock3d is not reaching it — res_x
passes x.channels as out_channels at :1051 and the guard is false, and encoder arm B
never enters ResnetBlock3d at all, since all four of its blocks are plain strided
CausalConv3d (:1060-1068). Rewritten to rest on the per-arm measurement and to say
what test_ltx2_vae.cpp:1327 already says: the coverage is real but PARTIAL.

Gate for ef83d9400

Release CPU build, -DVLLM_CPP_CUDA=OFF -DVLLM_CPP_METAL=OFF -DVLLM_CPP_VULKAN=OFF:

BUILD_EXIT=0                      (no "No space left" / "BFD assertion" line)
$ ctest --test-dir build-cpu -N
Total Tests: 401
$ ctest --test-dir build-cpu -j8 --output-on-failure
100% tests passed, 0 tests failed out of 401        CTEST_EXIT=0
suite before after exit Status
test_ltx2_vae 36 cases / 3039 assertions 36 / 3039 0 SUCCESS (comment-only)
test_ltx2_video 17 cases / 173 assertions 17 / 173 0 SUCCESS (comment-only)

Both baselines were measured on this same build dir before any edit. An earlier -j8 leg
saw test_engine_core_proc fail; it passed 5/5 on serial re-runs (--repeat until-fail:5)
and did not recur in the final leg — the known -j starvation, not a red.

Comment-only in the headers plus one docs/USAGE.md qualifier: no constant, golden, band,
or executable line changes, which is why the assertion counts are unchanged by construction.

doc-checkpoint range still fails on b0aa475a3 and d67f8125e — pre-existing L4 and
L7+L8 branch commits, flagged identically before any edit in this round and untouched by
it. scripts/agent-preflight.sh --staged is green. A campaign-integration item, not this
repair.

Round-4 repair (4f1ec6ec6) — the sweep found six more, at sites no round had enumerated

Issue: #560. Standard: #604.

An exhaustive sweep of every reachability claim in the tree measured 25 and found
six false. The good news first: the pin list in test_ltx2_vae.cpp that rounds 1–3 kept
repairing is correct in every entry and is untouched here. This round is
comments-and-records only — no constant, golden, band, tolerance or executable line
moves, and no gate is deleted.

Harness, because round 3 fabricated a result with the wrong one

Restoring a mutated file with cp/mv gives it an mtime older than the objects
built from the mutant, so ninja skips the rebuild and the previous mutation survives
into the next leg with BUILD_EXIT=0 and no diagnostic. Guards used on every one of the
ten legs here:

  • touch after every write and every restore;
  • the pristine tree proved green by a forced rebuild immediately before each mutant
    leg, with case and assertion counts asserted against the baseline;
  • restore verified back to the pre-mutation md5;
  • pre-vs-mutant binary md5 compared on every leg — the direct discriminator. All ten
    legs differed on the suite that red; none was binary-identical.

Baseline, and the post-edit result, identical by construction for a comments change:

suite baseline after exit
test_ltx2_vae 36 cases / 3039 assertions 36 / 3039 0
test_ltx2_pipeline 35 cases / 2358 assertions 35 / 2358 0
test_ltx2_text_encoder 17 cases / 3350 assertions 17 / 3350 0

The six, each reproduced on this tree before its text was touched

1. Ltx2ConvVideoDecoderConfig::pixel_norm_epstest_ltx2_vae.cpp claimed
1e-8 → 1e-6 "leaves every golden green".

test_ltx2_vae.cpp:1133: CHECK( err <= kLtx2GoldenTol ) is NOT correct!
  values: CHECK( 0.000169305 <= 5e-06 )
  logged: norm_eps-binding video decoder max|diff| = 0.000169305
36 cases: 34 passed, 2 failed | 3039 assertions: 3036 passed, 3 failed | exit 1

ltx2_video_vae.h has recorded this correctly since d45bcb5fb. The test file
contradicted its own header, so the test file is what moves.

2. "Unlike the decoder pair" — both halves of that pair are reachable. The audio
decoder's pixel_norm_eps 1e-6 → 1e-4 reds 5 goldens across three arms:

:419  CHECK( 0.0120053 <= 5e-06 )    audio decoder max|diff|
:607  CHECK( 0.00245912 <= 5e-06 )   causality_axis = 1
:607  CHECK( 0.00302449 <= 5e-06 )   causality_axis = 1
:607  CHECK( 0.00461239 <= 5e-06 )   causality_axis = 1
:639  CHECK( 0.0120053 <= 5e-06 )    padded audio decoder max|diff|
36 cases: 32 passed, 4 failed | 3039 assertions: 3033 passed, 6 failed | exit 1

All four PixelNorm epsilons are numerically gated, not two.

3. kLtx2UpsamplerNormEps — claimed invisible at BOTH ltx2_upsampler.h:62-65 and
test_ltx2_pipeline.cpp:1376-1380. At the class's own 100x bar, 1e-5 → 1e-3:

:1362  CHECK( 0.0289409 <= 5e-06 )   arm = PixelShuffle
:1362  CHECK( 0.0347079 <= 5e-06 )   arm = Rational2
:1362  CHECK( 0.0649014 <= 5e-06 )   arm = Rational1p5
35 cases: 34 passed, 1 failed | 2358 assertions: 2354 passed, 4 failed | exit 1

4. kLtx2UpsamplerNormGroups — called a member of the same class. 32 → 16 reds the
same three arms at 0.63738, 0.633718, 0.874346 (35 cases: 34/1; 2358: 2354/4).

5. kLtx2ConnectorRmsNormEps — claimed inert at BOTH ltx2_connector.h:65-69 and
test_ltx2_pipeline.cpp:1296-1301, on the reasoning "the fixture's rows are never
near-zero"
. That is not what rms_norm does with it: the epsilon is added to the mean
square
(ltx2_connector.cpp:60), so it perturbs every row it normalizes. 1e-6 → 1e-4:

:1614  CHECK( 0.0558581 <= 5e-06 )     arm = Split
:1614  CHECK( 0.104284 <= 5e-06 )      arm = Interleaved
:1614  CHECK( 0.140343 <= 5e-06 )      arm = Float64
:1614  CHECK( 0.000542641 <= 5e-06 )   arm = NoRegisters
:1614  CHECK( 0.0892045 <= 5e-06 )     arm = GatedNoBias
35 cases: 33 passed, 2 failed | 2358 assertions: 2352 passed, 6 failed | exit 1

6. kLtx2BlurKernelSize — called reachable "only through a default upstream never
passes explicitly"
. A default upstream never overrides is the shipped width, which
ltx2_upsampler.h:98-102 already said; the two records disagreed and the test file held
the wrong one. 5 → 3 reds 0.689782 on Rational1p5, and only that arm, because
BlurDownsample runs on the rational den (ltx2_upsampler.cpp:439) and 1.5 → {3, 2}
is the one covered scale with den != 1.

7. kLtx2Res2sSigmaUpClamp — a REASONING error, not a fixture gap. Both
ltx2_pipeline.h:174-178 and test_ltx2_pipeline.cpp:476-479 argued "eta <= 1 keeps
sigma_up <= sigma_next, so the clamp never binds"
. <= includes ==. step forms
sigma_up = sigma_next * eta (ltx2_pipeline.cpp:339), so at eta = 1 the two are equal
and min takes sigma_next * 0.9999 on every step — the clamp is not a fallback, it
is the only thing keeping the residual off zero exactly there. A 1% move, 0.9999 →
0.99, reds the Eta1 arm the suite already runs:

:463  CHECK( 0.086 <= 5e-06 )       arm = Eta1 index = 0
:463  CHECK( 0.130563 <= 5e-06 )    arm = Eta1 index = 1
35 cases: 34 passed, 1 failed | 2358 assertions: 2354 passed, 4 failed | exit 1

EtaHalf stays green because 0.5 * sigma_next is below the clamp; Eta1 index 2 stays
green because sigma_next == 0 returns the denoised prediction unchanged (:181-182).
The corrected note states the boundary, not just the verdict.

8. Secondary — kLtx2TextNormV1Eps / kLtx2TextNormV2Eps. Two blurbs in
ltx2_text_encoder.h (:66-67 "invisible to any golden built from random values";
:166-167 "Reachable only when…") contradicted the correct detailed note at
:150-163, which already warns of exactly this. Both are additive on an O(1) denominator
against a 1e-5 band, so both move ordinary random-value goldens at 100x:

V1 1e-6 -> 1e-4   :539 CHECK( 0.000524044 < 1e-05 )   `_norm_and_concat_padded_batch`
                  :684 CHECK( 7.53999e-05 < 1e-05 )   FeatureExtractorV1
                  :685 CHECK( 6.61612e-05 < 1e-05 )
                  17 cases: 14 passed, 3 failed | 3350: 3339 passed, 11 failed | exit 1
V2 1e-6 -> 1e-4   :571 CHECK( 0.00232971 < 1e-05 )    `norm_and_concat_per_token_rms`
                  :637 CHECK( 0.000344872 < 1e-05 )   FeatureExtractorV2
                  :638 CHECK( 0.000259042 < 1e-05 )
                  :783 CHECK( 0.00039053 < 1e-05 )    encoder -> conditioning hand-off
                  17 cases: 13 passed, 4 failed | 3350: 3338 passed, 12 failed | exit 1

The blurbs now agree with the note that was right all along.

9. One class-prose fix. ltx2_video_vae.h:106-107 said the tensor comparison "accepts
any value at all — including 0.0, and including one 100x off"
. False even of its own
remaining members: kLtx2RmsNorm2dEps at 1e-12 → 1.0 reds two encoder goldens.

:1746  CHECK( 0.000525832 <= 5e-06 )   video encoder (*_res) max|diff|
:1843  CHECK( 0.000525832 <= 5e-06 )   cropped video encoder max|diff|
36 cases: 33 passed, 3 failed | 3039 assertions: 3036 passed, 3 failed | exit 1

The per-entry verdict (1e-12 → 0.0 green) stands and is unchanged. "Never BINDS at the
shipped value" and "is not read" are different statements, and only the first was ever true.

The standard the corrected text is written to

Per #604: only a probe that FAILS to reach proves unreachable. A mutation that happens
not to move anything proves nothing, so every corrected claim now names its magnitude
and its arm. Magnitude escalation to the class's own 100x bar is what exposed (3), (5),
(6) and (9); a mere 1% exposed (7).

Every constant stays pinned. Reachability by a golden and a source-anchored pin are not
substitutes: a regeneration that moves a constant and its expected tensors together passes
every value comparison, and only the pin compares against upstream's own signature. The
corrected text says that, instead of claiming the pin is the only gate.

docs/USAGE.md gains the operator-facing half of the same finding, inside the pipeline
golden-regeneration recipe: which constants a regenerated ltx2_pipeline_goldens.inc now
carries, and why the pin cases are still not redundant.

Gate for 4f1ec6ec6

Release CPU build, -DVLLM_CPP_CUDA=OFF, 20 cores:

BUILD_EXIT=0                      (no "No space left" / "BFD assertion" line; df 85%)
$ ctest --test-dir build-ltx -N
Total Tests: 401
$ ctest --test-dir build-ltx -j 8 --output-on-failure
100% tests passed, 0 tests failed out of 401        CTEST_EXIT=0

An earlier -j 8 leg on a 399-test configure (-DVLLM_CPP_BUILD_EXAMPLES=OFF) saw
test_engine_core_proc fail; it passed serially and did not recur in the 401-test leg
above — the known -j starvation, not a red.

doc-checkpoint range still fails on b0aa475a3 and d67f8125e. Reproduced identically
at the untouched head with
python3 scripts/check-doc-checkpoint.py --base origin/main --head ef83d9400, so it
predates this round by 56 commits and belongs to the L4 / L7+L8 phases.
scripts/agent-preflight.sh --staged is green, and
check-doc-checkpoint.py --commit 4f1ec6ec6 is green.

Follow-up 8fef91748 — arm coverage, not the scale map

The blur entry 4f1ec6ec6 added read "1.5 → {3, 2} is the one supported scale of the
three with den != 1"
, which as written claims the supported-scale MAP has one non-unit
denominator. It does not: 0.75 → {3, 4} (ltx2_upsampler.cpp:296) would reach
kLtx2BlurKernelSize just as well. The true statement is narrower — of the three arms the
suite runs (PixelShuffle 2.0 non-rational, Rational2 2.0 → den 1, Rational1p5 1.5 → den 2,
per ltx2_pipeline_goldens.inc:1335, :1568, :1801), only Rational1p5 has den != 1. That
is arm coverage, and 0.75 is an uncovered scale rather than a nonexistent one; the
corrected text names it so the gap stays visible. Comment text only, in one test file.

BUILD_EXIT=0     ctest -N: 401     ctest -j 8: 401/401 passed, CTEST_EXIT=0
test_ltx2_pipeline   35 cases / 2358 assertions, exit 0   (unchanged)

🤖 Generated with Claude Code

mudler added 7 commits August 13, 2026 00:30
…half (#560)

Closes the two LOW findings from the phase-L11 review, and the pre-existing
half of the first one that L4 left behind.

THE INVISIBLE-CONSTANT CLASS, FOURTH RECURRENCE, inside a change that cites it.
`Ltx2AudioEncoderConfig::norm_eps` was absent from the pin list whose own
comment says "Adding a new constant without adding it to this list reopens the
hole", and `Ltx2AudioDecoderConfig::norm_eps` had the identical hole from L4.
Mutating BOTH from 1e-6 to 1e-4, a 100x change, left every golden green:

    [doctest] test cases:   33 |   33 passed | 0 failed | 0 skipped
    [doctest] assertions: 2602 | 2602 passed | 0 failed |
    [doctest] Status: SUCCESS!

The cause is worse than inertness. Every audio arm in the suite runs
`norm_type = kPixel`, so `ApplyNorm` never enters the GroupNorm branch and the
constant is never READ on any executed path. A pin alone would have recorded
that fact rather than removed it, so this takes the option spec section 7.0(a)
actually asks for: two new GOLDEN ARMS that execute the branch.

`norm_type = group` is not hypothetical. It is the DEFAULT of both
`AudioEncoder.__init__` and `AudioDecoder.__init__` (audio_vae.py:81, :295), it
is the other half of `build_normalization_layer` (normalization.py:56-57), and
it is legal wherever `causality_axis` is `none`, which is exactly what
`ResnetBlock.__init__` permits (resnet.py:130-131). A checkpoint declaring it
would have run a 100x-wrong stabilizer and still produced a spectrogram.

Two reduced dimensions move on those arms and both are FORCED, not chosen:
`ch` becomes 32 because `build_normalization_layer` hardcodes `num_groups=32`
and torch's GroupNorm refuses any channel count 32 does not divide, and
`z_channels` becomes 16 because `PerChannelStatistics` indexes the patchified
(c, f) axis so `z_channels * mel_bins` must equal `ch`. Nothing else changes.

The arms are numerical, not assertional. Re-running the same 100x mutation now
moves the decoder golden by 1.13e-3 and the encoder golden by 5.18e-3 against a
5e-6 band, i.e. 225x and 1036x over. The pin is added as well, because a pin
catches the edit a golden cannot: swapping 1e-6 for the video VAE's 1e-8 while
regenerating would move the goldens and the arms would follow it. The same
sweep found `Ltx2ConvVideoEncoderConfig`'s two epsilons missing from the list
as well; they are pinned here too.

A REFUSAL WHOSE STATED REASON WAS FALSE. The keyframe / reference refusal still
read "phase L4 ported the DECODER only". L11 ported the encoder. The refusal's
effect is unchanged and correct, but a stale reason sends the next reader to
build something that already exists, so it now names what is actually missing:
no VAE_ENCODER_COMFY_KEYS_FILTER / VideoEncoderConfigurator loader path puts
encoder weights in memory, and `ImageConditioner.resolve_crf`'s H.264
`default_image_crf` round trip is not ported. The test that guards it was
weak enough to accept either message, so it now asserts both of those facts and
goes RED on the old text.

Gates, all on a clean Release CPU build:
  test_ltx2                29/1615  SUCCESS   (unchanged)
  test_ltx2_vae            35/2996  SUCCESS   (was 33/2602; +2 group-norm arms)
  test_ltx2_text_encoder   17/3350  SUCCESS   (unchanged)
  test_ltx2_pipeline       35/2358  SUCCESS   (unchanged)
  test_ltx2_loader         20/2363  SUCCESS   (unchanged)
  test_ltx2_video          17/172   SUCCESS   (was 17/170; +2 refusal assertions)
  test_ops_attention_cross  9/32    SUCCESS   (unchanged)
  test_video_engine        11/254   SUCCESS   (unchanged)
  test_capi                55/505   SUCCESS   (unchanged)

The goldens regenerate byte-identically outside the two new sections: the
generator was run against the pinned oracle before and after, and the diff is
276 added lines and zero removed.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
…560)

Sweeping the class rather than only the two named constants was the point of
#560, and it paid: a 20-constant mutation sweep over every stabilizing epsilon,
clamp and floor in the LTX-2.5 files turned up one more constant that NO arm can
see, and one gap in a case's own claim.

`Ltx2DitParams::norm_eps` is INVISIBLE. Mutating it 1e-6 -> 1e-4, a 100x change,
left all six LTX suites green: 29/1615, 35/2996, 17/3350, 35/2358, 20/2363,
17/172, all SUCCESS. The reason is not that the constant is unused — it feeds
the q/k RMSNorm (transformer/attention.py:505-506) and every AdaLN — but that
every arm in test_ltx2 passes it EXPLICITLY through `ReducedParams`, so nothing
reads the FIELD DEFAULT.

That default is live code. `ReducedConfig()` carries no `norm_eps` key, which is
exactly the shape of a checkpoint that omits it, and `ParseLtx2DitParams` then
falls back to the field, mirroring upstream's own
`config.get("norm_eps", 1e-06)` (transformer/model_configurator.py:54, 124, 181).
So the pin goes where the default actually binds, in the parse case, with the
override direction gated alongside it rather than in a list far from its use.

The override check is a plain block and not a SUBCASE on purpose: doctest
re-enters the whole case body once per subcase, so adding one would multiply
every assertion above it and move this suite's recorded count for a reason that
has nothing to do with coverage.

`Ltx2AudioEncoderConfig::pixel_norm_eps` was the second gap, and a smaller one:
it is reachable by three goldens, so it was never silent, but "the two PixelNorm
epsilons stay different" only ever held the DECODER pair, and phase L11 added an
encoder pair with the same 1e-6 / 1e-8 split for the same reason. The case now
holds every config that has the field. Proven by the mutation it exists to
catch: unifying the audio encoder onto the video VAE's 1e-8 turns four cases red.

Gates on a clean Release CPU build:
  test_ltx2                29/1625  SUCCESS   (was 29/1615; +10, the DiT pin)
  test_ltx2_vae            35/2998  SUCCESS   (was 35/2996; +2, the encoder pair)
  test_ltx2_text_encoder   17/3350  SUCCESS   (unchanged)
  test_ltx2_pipeline       35/2358  SUCCESS   (unchanged)
  test_ltx2_loader         20/2363  SUCCESS   (unchanged)
  test_ltx2_video          17/172   SUCCESS   (unchanged)

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
…ate was 401 (#560)

Repairs the review findings on this branch. Two were blocking: a constant this
change classified as correctly unreachable is live upstream, and the gate line it
reported was wrong in a way that would have concealed a dropped test.

A CONSTANT CALLED UNREACHABLE THAT UPSTREAM READS ON EVERY ARM.
`Ltx2ConvVideoDecoderConfig::norm_eps` was recorded as "pinned, correctly
unreachable — upstream discards the value" on the reasoning that the decoder runs
`norm_layer = PIXEL_NORM`. That reasoning does not hold. `ResnetBlock3D.__init__`
builds `norm3 = nn.GroupNorm(num_groups=1, num_channels=in_channels, eps=eps)`
whenever `in_channels != out_channels` (video_vae/resnet.py:93-97), REGARDLESS of
`norm_layer`, and `forward` applies it to the residual (resnet.py:178). Every
`res_x_y` block reads the constant, and the shipped section-5 fixture has one, at
`up_blocks.4.norm3`. Our port reads it at the same place
(ltx2_video_vae.cpp:400-405).

What was actually true was a sensitivity statement about ONE FIXTURE. norm3
divides by `sqrt(var + eps)` over all of (C, T, H, W); five blocks deep that
variance is ~0.2, so 1e-6 -> 1e-4 moves the golden 1.8e-6, under the 5e-6 band,
while 1e-6 -> 1.0 moves it 1.6e-2. A 100x error passed because the denominator
was large. Measured on the oracle, not inferred:

    eps 1e-6 -> 1e-05    max|diff| = 5.960e-07   green
    eps 1e-6 -> 0.0001   max|diff| = 1.848e-06   green
    eps 1e-6 -> 0.001    max|diff| = 1.687e-05   RED
    eps 1e-6 -> 1        max|diff| = 1.643e-02   RED
    eps 1e-6 -> 0        max|diff| = 5.364e-07   green

A pin records that hole; it does not remove it. So section 5d is a new GOLDEN ARM
built to remove it: ONE `res_x_y` block, so norm3 sits directly behind conv_in,
and a latent at a tenth of the usual scale, so the variance the epsilon competes
with is ~5e-3 rather than ~0.2. The generator asserts the arm is sensitive before
emitting it, and emits what it measured (`kLtx2VideoDecEpsZeroMove` = 1.024e-4)
so the C++ side gates the sensitivity instead of narrating it.

RED, mutating the epsilon AT ITS POINT OF USE so the pins stay silent and only
the numerics speak. 1e-6 -> 1e-5, the smallest realistic typo:

    TEST CASE:  ltx2 vae: the video decoder's norm_eps is gated where it BINDS
      values: CHECK( 0.000919342 <= 5e-06 )
    [doctest] test cases:   36 |   35 passed | 1 failed | 0 skipped
    [doctest] Status: FAILURE!

and removing the epsilon entirely, which no other arm in the file can see:

    TEST CASE:  ltx2 vae: the video decoder's norm_eps is gated where it BINDS
      values: CHECK( 0.000102103 <= 5e-06 )
    [doctest] test cases:   36 |   35 passed | 1 failed | 0 skipped
    [doctest] Status: FAILURE!

One case fires out of 36 in both, which is the point: the five arms that already
execute norm3 stay green through a 10x error and through deleting the constant.
GREEN after restoring: 36/3039 SUCCESS. The 1.021e-4 the port moves under eps=0
agrees with the 1.024e-4 the oracle measured, so both sides read the constant in
the same place.

The goldens regenerate byte-identically outside the new section: 60 added lines,
zero removed.

THE GATE WAS 401, AND THE RECONCILIATION WAS INVENTED. The PR body claimed
"399/399 ... (399 rather than 401 is this CPU-only configuration's registered
count, not a dropped test)". That is false. On a clean Release CPU build of this
tree `ctest -N` reports `Total Tests: 401`, and `ctest -j4` reports
`100% tests passed, 0 tests failed out of 401`. Nothing was dropped — but the
explanation was manufactured rather than checked, and it is exactly the shape of
reasoning that would have hidden a real drop. This branch adds no CMake test
target, so the registered count is unchanged from the base by construction.

A SIXTH CONSTANT, AND AN HONEST LABEL FOR IT. `Ltx2AttentionArgs::norm_eps`
(ltx2.h) is the q/k RMSNorm epsilon — `Attention.__init__`'s
`norm_eps: float = 1e-6` (attention.py:485), handed to both RMSNorms
(attention.py:505-506). Every construction of the struct assigns it before use:
ltx2_dit.cpp:188, :244, :280, :338, :366, ltx2_connector.cpp:253, and each of the
suite's own arms. Mutating the default 1e-6 -> 1.0, a 10^6 change, leaves every
LTX suite green except the new pin:

    TEST CASE:  ltx2 dit: Ltx2AttentionArgs::norm_eps is a LATENT default
      values: CHECK( 1 == Approx( 1e-06 ) )
    [doctest] test cases:   30 |   29 passed | 1 failed
    [doctest] Status: FAILURE!
    test_ltx2_vae 36/36, test_ltx2_pipeline 35/35, test_ltx2_loader 20/20,
    test_ltx2_video 17/17, test_ltx2_text_encoder 17/17, test_ltx2_device 13/13
    — all SUCCESS under the same mutation.

That is not the invisible-epsilon story the others tell. Those are read and never
bind; this one is never READ, so no fixture at any scale can reach it. It is a
latent trap — the value a future call site inherits on the day someone adds one
and forgets the assignment. A pin is the only instrument that can hold it, and
the comment says so rather than presenting a pin as coverage.

A DEFAULT THAT RAISES. Five places claimed `norm_type = group` "is the DEFAULT of
both `AudioEncoder.__init__` and `AudioDecoder.__init__`". The declaration is
real (audio_vae.py:82, :294) but the sentence is misleading: the paired default
on the very next line is `causality_axis = WIDTH` (audio_vae.py:83, :295), and
`ResnetBlock` refuses that combination with `ValueError: Causal ResnetBlock with
GroupNorm is not supported` (audio_vae/resnet.py:130-131). Constructing either
class on pure defaults RAISES — verified by construction against the pin, not
read off the signature. All five now say what is true: a group-norm checkpoint is
one that declares `causality_axis: none` alongside it, which is what the arms
run. The two anchors were also off by one in opposite directions (`:81` -> `:82`,
`:295` -> `:294`); both fixed.

Three smaller inaccuracies in the same sweep. `num_groups=32` was described as
hardcoded in `build_normalization_layer`; it is that function's own keyword-only
DEFAULT (normalization.py:44) forwarded at :56, and what makes it binding is that
no audio_vae call site passes one — `eps=1e-6` is the literal. The generator's
"nothing else changes, so a diff against the pixel arms is exactly the norm" was
wrong about the causality axis, which changes too and must. And the keyframe
refusal overstated `resolve_crf`: it only FILLS IN `default_image_crf` for a
conditioning that left `crf=None` (blocks.py:977-983); the H.264 round trip is
`preprocess` (media_io/decode.py:413-435), and it is SKIPPED at `crf == 0`, which
a caller may pass explicitly. The message and docs/USAGE.md now say that.

Gates, clean from-scratch Release CPU build (-DVLLM_CPP_CUDA=OFF), zero warnings:
  ctest -N               401 registered
  ctest -j4              100% tests passed, 0 failed out of 401
  test_ltx2              30/1627  SUCCESS  (was 29/1625; +1 case, the args pin)
  test_ltx2_vae          36/3039  SUCCESS  (was 35/2998; +1 case, the eps arm)
  test_ltx2_device       13/498   SUCCESS
  test_capi              55/505   SUCCESS
  test_ltx2_video        17/172   SUCCESS
  test_ltx2_pipeline     35/2358  SUCCESS
  test_ltx2_loader       20/2363  SUCCESS
  test_ltx2_text_encoder 17/3350  SUCCESS

`doc-checkpoint range` still fails on `b0aa475a3` and `d67f8125e`, both of which
predate this branch and are untouched by it.

.agents/specs/ltx-2-5.md is operator-owned and not edited here; its
classification table was corrected on row/MODEL-DIFFUSION-LTX25 at `10f2b7451`.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
…icts (#560)

This PR exists to stop constants carrying a wrong reachability verdict, and it
shipped three of its own. The deliverable is an accurate record, so an inaccurate
record here IS the defect. Every claim below was re-measured on this tree before
the text it falsifies was touched; nothing about the norm_eps arm or the upstream
revision pin changes.

TWO CONSTANTS LISTED AS INVISIBLE ARE NUMERICALLY GATED. The pin case opened with
"Each of these was mutated with every golden staying green", which is false for
both encoder entries. Mutating the FIELD DEFAULTS, which is what every arm runs
because no arm overrides them:

    Ltx2ConvVideoEncoderConfig::norm_eps       1e-6 -> 1e-4
      RED 4.38839e-05 vs the 5e-6 band, on 2 goldens
      ("the video ENCODER (*_res family)", "the video encoder CROPS a frame
       count that is not 1 + k*factor")
    Ltx2ConvVideoEncoderConfig::pixel_norm_eps 1e-8 -> 1e-6
      RED on 4 goldens: 1.02744e-05, 1.02744e-05, 8.10623e-06, and
      0.000175595 on "(strided convs, per_channel, reflect)"

The cause is the one this PR already established for the decoder, reaching the
other half through the SAME LINE. The encoder shares ResnetBlock3d --
ltx2_video_vae.cpp:1051,1056 call what the decoder calls at :693,700 -- so it
reads norm3 at :405 for exactly the reason F-1 gave. Forcing :405 to 1.0 reds
those two encoder goldens at 0.150858, which is what IDENTIFIES norm3 as the
reader rather than inferring it: norm_layer is kPixelNorm on both encoder arms,
so neither ApplyNorm nor conv_norm_out (:1081-1087) enters a GroupNorm branch.
Arm B has no res_x_y and so no norm3, and stays green -- the coverage is real but
partial, which is why the pins stay.

Coverage was BETTER than recorded, not worse, and that is still a defect: a list
whose membership claim is wrong in the safe direction is a list nobody can trust
in the unsafe one. The class statement is now per-entry, quantified, and says the
mutation was run, because this case has carried a wrong verdict twice.

THE NEW ARM FALSIFIED A LINE THIS PR LEFT STANDING. ltx2_video_vae.h recorded
`Ltx2ConvVideoDecoderConfig::pixel_norm_eps 1e-8 -> 1e-6 green` under "EVERY
golden staying green". With section 5d present that mutation REDS at 1.69305e-04:
the low-scale latent built to make norm_eps a first-order term made its neighbour
one too. The fixture that closed one hole closed another, and the line claiming
otherwise survived the change that refuted it.

THE ENCODER HALF GETS THE SAME NOTE AS THE DECODER. ltx2_video_vae.h gained a
full "norm3 is the reason this is LIVE on a PixelNorm checkpoint too" note;
ltx2_video_vae_encoder.h still said only "a field here only so the gate can pin
it". True, and incomplete for the identical code path. One line cannot be live
for one caller and dead for the other.

THE QUALIFIER F-8 ADDED IS NOW GATED. The refusal test checked only
VAE_ENCODER_COMFY_KEYS_FILTER and default_image_crf, so deleting the `crf == 0`
qualifier again would not go red. Upstream `preprocess` returns the image
untouched at crf == 0 (media_io/decode.py:427, in :413-435), so "re-compresses
before encoding" is only true of a nonzero resolved CRF; naming the round trip
without its exception overstates what is unported. Proven RED by removing the
qualifier from the message: 16 passed / 1 failed, the one being the new CHECK.

ONE BAND, ONE DEFINITION. gen-ltx2-vae-goldens.py hardcoded `10 * 5e-6` while the
suite applies kLtx2GoldenTol. Two definitions of one number in two languages: a
widened C++ band would leave the generator certifying arms against a band nobody
uses. It now PARSES kLtx2GoldenTol from the suite and dies unless it finds exactly
one definition. And docs/USAGE.md:335 had grown to 119 chars inside a 79-column
paragraph; re-wrapped, no wording changed.

Gate on this tree, CPU-only Release, clean rebuild: BUILD_EXIT=0, no ENOSPC or
BFD line, ctest -N 401, ctest -j4 400/401 with test_engine_core_proc the known -j
flake, PASS on a serial re-run. test_ltx2_vae 36 cases / 3039 assertions (comment
-only, unchanged from before). test_ltx2_video 17 cases / 173 assertions, up from
172 by the added CHECK.

Issue: #560 (campaign #435)

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
… list (#560)

Round 3 closed every repair from round 2 and failed on one new instance of the
same defect: a record calling a constant unreachable when a golden reds on it.
One entry over, the list still said green.

THE BWE MEL LOG CLAMP IS NOT INVISIBLE. ltx2_video_vae.h:108-110 carried

    kLtx2BweMelLogClamp                    1e-5 -> 1e-8   green

under "these three left EVERY golden green". Reproduced on this tree before
touching the text, by mutating ltx2_audio_vae.h:212:

    test_ltx2_vae.cpp:1400: CHECK( err <= kLtx2GoldenTol ) is NOT correct!
      values: CHECK( 0.144965 <= 5e-06 )
      logged: saturated-clamp BWE max|diff| = 0.144965
    36 cases: 34 passed, 2 failed | 3039 assertions: 3037 passed, 2 failed
    exit code 1

The second failure is the constant assertion at :1279. Restored byte-for-byte
afterwards, md5 fe861901ef8f18fbbc9e6caedf513d1b before and after.

The saturating arm that reds it, "ltx2 vae: the BWE mel log clamp is gated where
it actually binds", landed at 93329b1 -- and `git log -S "left EVERY golden
green"` returns only d45bcb5, so the phrase was written over an arm that
already existed. Two places in the tree already said so: this same header 26
lines later, and test_ltx2_vae.cpp:1274-1278 ("until the saturating arm below").
The list is the surface a reader trusts, so the list is what was wrong.

The entry is moved out with the number that proves it, exactly as round 2 moved
pixel_norm_eps. What made it look invisible is SCALE, not the constant's nature:
the ordinary arm's raw mel minimum is ~4.4e-3 and never approaches the floor, so
the reachable arm attenuates mel_basis by 1e-4 until every bin lands under it and
asserts the saturated-bin count rather than assuming it.

THE SAME FALSE CLAIM SAT AT THE CONSTANT'S OWN DEFINITION. ltx2_audio_vae.h:206
declared it "the member of the invisible-constant class" and stated the 1e-5 ->
1e-8 mutation "leaves every tensor golden green". Both sentences shipped in
93329b1, the very commit that added the golden refuting them. Fixing the list
while leaving the declaration would have left round 4 the identical finding one
file over, so it is corrected here. Its upstream citation was also off by one --
the clamp is vocoder.py:515, not :516; :516 is the `return`. Verified at the
pinned checkout, which is what test_ltx2_vae.cpp:1273 already cited correctly.

THE CRF CITATION POINTED AT THE WRONG EARLY RETURN. test_ltx2_video.cpp:668 read
"media_io/decode.py:413-435, the early return at :427". At LTX-2 fd4ded7f,
`if crf == 0:` is :425 and `return image` is :426; :427 is
`if min(image.shape[0], image.shape[1]) < 2:`, a different early return for
degenerate image size. The range and the behavioural claim were both right, so
only the pinpoint moves -- but this is the worst place to be off by two, because
a reader following it lands on the size guard and concludes the crf == 0 claim is
unsupported. Now cited as :425-426 and naming :427-428 as the guard it is not.

That size guard is also a second case where upstream does NOT re-compress, and
docs/USAGE.md:334 claimed the round trip happens "whenever the resolved CRF is
not 0" with no other exception. Qualified there too.

A SHARED LINE IS NOT AN ARGUMENT FOR LIVENESS. ltx2_video_vae_encoder.h:130-132
argued the encoder's norm_eps is live because "It is ONE line in the port for
both halves ... so it cannot be live for one and dead for the other". The
conclusion holds -- both halves were measured live -- but the inference does not,
and round 3's :405 -> 1.0 probe is the counterexample: :405 sits behind the
`input.channels != out_channels` guard at :400, so even entering ResnetBlock3d is
not reaching it. `res_x` passes `x.channels` as `out_channels` at :1051 and the
guard is false; encoder arm B never enters ResnetBlock3d at all, since all four
of its blocks are plain strided CausalConv3d (:1060-1068). Rewritten to rest on
the per-arm measurement, and to say what test_ltx2_vae.cpp:1327 already says --
the coverage is real but PARTIAL, which is what the pin is still for.

Comment-only in the headers plus one doc qualifier; no constant, golden, band or
executable line changes, and the assertion counts are unchanged by construction.

Gate on this tree, CPU-only Release: BUILD_EXIT=0, no ENOSPC or BFD line,
ctest -N 401, ctest -j8 401/401 CTEST_EXIT=0. test_ltx2_vae 36 cases / 3039
assertions exit 0; test_ltx2_video 17 cases / 173 assertions exit 0 -- both
identical to the pre-edit baseline measured on the same build dir.

`doc-checkpoint range` still fails on b0aa475 and d67f812. Both are
pre-existing L4 and L7+L8 branch commits, both flagged identically before any
edit in this change, and neither is touched here; it is a campaign-integration
item, not this repair.

Issue: #560 (campaign #435)

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
, #604)

An exhaustive sweep of every reachability claim in the tree measured 25 and found
six false, at sites no earlier round on this branch enumerated. The pin list in
test_ltx2_vae.cpp that rounds 1-4 kept repairing is correct in every entry and is
untouched. Nothing here weakens a bound, adds tolerance or deletes a gate: the
suites keep their exact case and assertion counts.

Every number below was reproduced on this tree before the text it falsifies was
edited, one mutation per leg, each leg preceded by a FORCED rebuild of the
pristine tree proved green and followed by a restore verified back to the
pre-mutation md5. Binary md5s were compared pre vs mutant on every leg and
differed on all ten, which is the discriminator for the ninja-mtime leak that
would otherwise let one leg's mutation survive into the next.

1. Ltx2ConvVideoDecoderConfig::pixel_norm_eps. test_ltx2_vae.cpp said 1e-8 ->
   1e-6 "leaves every golden green". It REDS "the video decoder's norm_eps is
   gated where it BINDS" at CHECK( 0.000169305 <= 5e-06 ). ltx2_video_vae.h has
   recorded this correctly since d45bcb5; the test file contradicted its own
   header, so the test file is what moves.

2. "Unlike the decoder pair". Both halves of that pair are reachable. The audio
   decoder's pixel_norm_eps 1e-6 -> 1e-4 REDS 5 goldens across three arms --
   0.0120053, 0.00461239, 0.00302449, 0.00245912, 0.0120053 -- and the video half
   reds per (1). All four PixelNorm epsilons are numerically gated.

3. kLtx2UpsamplerNormEps, claimed invisible at BOTH ltx2_upsampler.h and
   test_ltx2_pipeline.cpp. At the class's OWN 100x bar, 1e-5 -> 1e-3 REDS all
   three arms of "the latent spatial upsampler reproduces upstream": PixelShuffle
   0.0289409, Rational2 0.0347079, Rational1p5 0.0649014.

4. kLtx2UpsamplerNormGroups, called a member of the same class. 32 -> 16 REDS the
   same three arms at 0.63738, 0.633718, 0.874346.

5. kLtx2ConnectorRmsNormEps, claimed inert at BOTH ltx2_connector.h and
   test_ltx2_pipeline.cpp on the reasoning "the fixture's rows are never
   near-zero". That is not what rms_norm does with it: the epsilon is added to the
   MEAN SQUARE, so it perturbs every row. 1e-6 -> 1e-4 REDS 5 connector arms --
   Split 0.0558581, Interleaved 0.104284, Float64 0.140343, NoRegisters
   0.000542641, GatedNoBias 0.0892045.

6. kLtx2BlurKernelSize, called reachable "only through a default upstream never
   passes explicitly". A default upstream never overrides IS the shipped width,
   which ltx2_upsampler.h:98 already said -- the two records disagreed and the
   test file held the wrong one. 5 -> 3 REDS the upsampler's Rational1p5 arm at
   0.689782, and only that arm, because BlurDownsample runs on the rational `den`
   (ltx2_upsampler.cpp:439) and 1.5 -> {3, 2} is the one covered scale with
   den != 1.

7. kLtx2Res2sSigmaUpClamp -- a REASONING error, not a fixture gap, and the one
   worth reading. Both ltx2_pipeline.h and test_ltx2_pipeline.cpp argued "eta <= 1
   keeps sigma_up <= sigma_next, so the clamp never binds". <= includes ==. `step`
   forms sigma_up = sigma_next * eta (ltx2_pipeline.cpp:339), so at eta = 1 the
   two are equal and `min` takes sigma_next * 0.9999 on EVERY step -- the clamp is
   not a fallback, it is the only thing keeping the residual off zero exactly
   there. A 1% move, 0.9999 -> 0.99, REDS the Eta1 arm the suite already runs, at
   0.086 (index 0) and 0.130563 (index 1); EtaHalf stays green because 0.5 *
   sigma_next is below the clamp, and Eta1 index 2 stays green because sigma_next
   == 0 returns the denoised prediction unchanged (:181-182). The corrected note
   states the boundary, not just the verdict.

8. kLtx2TextNormV1Eps / kLtx2TextNormV2Eps. Two blurbs in ltx2_text_encoder.h
   ("invisible to any golden built from random values", "Reachable only when...")
   contradicted the CORRECT detailed note 14 lines above one of them, which
   already warns of exactly this. Both epsilons are additive on an O(1)
   denominator against a 1e-5 band. V1 1e-6 -> 1e-4 REDS
   "`_norm_and_concat_padded_batch`, both padding sides" at 0.000524044 and
   "FeatureExtractorV1" at 7.53999e-05 / 6.61612e-05. V2 likewise REDS
   "`norm_and_concat_per_token_rms`" at 0.00232971, carrying into
   "FeatureExtractorV2" and the hand-off at 0.000344872 / 0.000259042 /
   0.00039053. The blurbs now agree with the note.

9. The class prose itself. ltx2_video_vae.h:106-107 said the tensor comparison
   "accepts any value at all -- including 0.0, and including one 100x off". False
   even of its own remaining members: kLtx2RmsNorm2dEps at 1e-12 -> 1.0 REDS "the
   video ENCODER (*_res family)" and "the video encoder CROPS a frame count that
   is not 1 + k*factor", both at 0.000525832. The per-entry verdict (1e-12 -> 0.0
   green) stands and is unchanged. "Never BINDS at the shipped value" and "is not
   read" are different statements, and only the first was ever true.

Per #604 the standard is now explicit in the prose: only a probe that FAILS TO
REACH proves unreachable, a mutation that happens not to move anything proves
nothing, and every corrected claim therefore names its magnitude and its arm.
Magnitude escalation to the class's own 100x bar is what exposed (3), (5), (6)
and (9); a mere 1% exposed (7).

Every constant stays PINNED. Reachability by a golden and a source-anchored pin
are not substitutes: a regeneration that moves a constant and its expected tensors
together passes every value comparison, and only the pin compares against
upstream's own signature. The corrected text says that instead of claiming the
pin is the only gate.

docs/USAGE.md gains the operator-facing half of the same finding, in the pipeline
golden regeneration recipe: which constants a regenerated ltx2_pipeline_goldens.inc
now carries, and why the pin cases are still not redundant.

GATE, CPU-only Release, VLLM_CPP_CUDA=OFF, 20 cores:
  BUILD_EXIT=0, no "No space left" or "BFD assertion" in any build log, df 87%.
  ctest -N            399 tests
  ctest -j 8          398/399 passed; test_engine_core_proc failed and PASSES
                      serially (known -j flake), so the gate is green.
  test_ltx2_vae            36 cases / 3039 assertions, exit 0
  test_ltx2_pipeline       35 cases / 2358 assertions, exit 0
  test_ltx2_text_encoder   17 cases / 3350 assertions, exit 0
Counts are byte-identical to the pre-edit baseline, which is the correct result
for a comments-and-records change: nothing was added, removed or relaxed.

The pre-existing `doc-checkpoint range` failure on b0aa475 and d67f812 is
NOT repaired here. It reproduces identically at the untouched head ef83d94
(`check-doc-checkpoint.py --base origin/main --head ef83d94`), predates this
work by 56 commits, and belongs to the L4 / L7+L8 phases rather than to this
scoped repair.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
…, #604)

A precision nit in the note 4f1ec6e added, in the one place it matters: the
deliverable of this branch is an accurate record, so a loose sentence in it is
the same defect one size down.

The blur-width entry read "1.5 -> {3, 2} is the one supported scale of the three
with den != 1". Read as written that says the SUPPORTED-SCALE MAP has one entry
with a non-unit denominator, which is false -- 0.75 -> {3, 4}
(ltx2_upsampler.cpp:296) has one too and would reach kLtx2BlurKernelSize just as
well. What is actually true is narrower: of the three arms the suite runs
(PixelShuffle at 2.0 non-rational, Rational2 at 2.0 -> den 1, Rational1p5 at 1.5
-> den 2, per ltx2_pipeline_goldens.inc:1335, :1568, :1801), only Rational1p5 has
den != 1, so only it reaches the constant. That is ARM COVERAGE, and 0.75 is an
uncovered scale rather than a nonexistent one.

Stating it the loose way would have left a reader believing the map cannot grow
another reachable arm, which is exactly the kind of inference #604 asks these
notes to stop making. The corrected text names 0.75 explicitly so the gap is
visible rather than argued away.

Comment text only, in one test file.

GATE, CPU-only Release, VLLM_CPP_CUDA=OFF:
  BUILD_EXIT=0, no "No space left" or "BFD assertion", df 85%.
  ctest -N     401 tests
  ctest -j 8   100% tests passed, 0 failed out of 401   CTEST_EXIT=0
  test_ltx2_pipeline   35 cases / 2358 assertions, exit 0
Counts unchanged from 4f1ec6e and from the pre-edit baseline.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
@localai-bot
localai-bot merged commit 310fa16 into row/MODEL-DIFFUSION-LTX25 Aug 13, 2026
@localai-bot

Copy link
Copy Markdown
Collaborator Author

Landed on row/MODEL-DIFFUSION-LTX25 at 310fa1688 — verified with git merge-base --is-ancestor. Operator gate rerun on the merged tree: clean Release build, BUILD_EXIT=0, zero warnings, enospc 0, ctest 414/414, exit 0, no flake to re-run.

Closing rather than merging: its base branch has moved on and the work is already on the campaign branch, reaching main through #437.

Worth recording what this row cost and what it bought, because it is the campaign's clearest methodological result.

Four scoped review rounds, then one exhaustive sweep. Rounds 1-4 each repaired the pin list in test_ltx2_vae.cpp and stopped — so the same defect class survived at five sites nobody had enumerated (test_ltx2_pipeline.cpp, ltx2_upsampler.h, ltx2_connector.h, ltx2_pipeline.h, and one case in test_ltx2_vae.cpp outside the list). Enumerating all 25 reachability claims and mutating each found six false in a single pass. The method mattered far more than the effort.

The best finding is a reasoning error rather than a fixture gap. kLtx2Res2sSigmaUpClamp was documented as "never binds on a well-formed schedule, so no value comparison can see it". Ltx2Res2sStep forms sigma_up = sigma_next * eta, so at eta = 1 the min takes sigma_next * 0.9999 on every step — the clamp is the only thing keeping residual = sqrt(next^2 - up^2) off exactly zero, and a 1% change reds two goldens on an arm the suite already ran.

Two sub-species this row named, now generalized in #604:

  • Magnitude sensitivity mistaken for unreachability. Several claims rested on a mutation at the minimum magnitude leaving goldens green; at the class's own stated 100x bar they red hard.
  • An assertion true by construction, which mutation testing structurally cannot catch because there is no code under test in it.

Also fixed here: an inaccuracy in the record that had shipped in the same commit as the golden refuting it, and an off-by-two citation pointing at a neighbouring early return — the worst place to be wrong, since a reader following it concludes the claim is unsupported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants